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ABSTRACT _ 

_ _ The general purposes of this study were to _ 

;tnvestigate_the factorial validity of the Comprehensiy Tests of 
.Basic Skills^ JCTBSjr Heading Comprehension subtest ^ and to study the 
item^dif f iculties_in order to determine areas of strengths and 
weaknesses of sixth and seventh grade Elementary Secondary Education 
Act (ESEA) Title I students. A sample of 5B1 Title I students was 
administered the test. Using Bartlett's test^ it was determined that 
the item- intercorrelatibh matrices were worth factor analyzing. The 
CTBS did not have a strong factor structure. Rather than being _ 
unidimehsionalr the test results appeared to be multidimensional. 
This study suggests that the _ results from testing Title I students, 
with standardized tests should be interpreted cautiously. The total 
score for each student is far from representing any one construct and 
the error variance in the testing is considerable. (Author/BW) 
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The Factorial Structure and Item Cbiiipdsitlbii 
of the Comprehensive Tests of Basic Skills* Reading 
Gomprehehsioh Test, When Administered to ESEA Title I 
Sixth and Seventh Grade Students 

Porter, Schmidt, Floden and Freeman (1978) advocate an "Intimate 
familiarity" with the measures used to estimate a program's effect. This 
familiarity would assist in the match of test content with program 
bbjeetives. The need for program valid achievement testing has been 
emphasized by many authors (Porter, et al , 1978; Blbbm, Hastings and 
Madaus, 1971; Nannally and Wilson, 1975; and Shoemaker, 1975). 

One approach to obtaining an intimate familiarity with an 
achievement test, is to scrutinize the Items and investigate the 
factorial validity of the test. Achievement tests are purportedly 
constructed to be uhiditnehsibhal and this is supported by high internal 
reliability estimates.^ Bhe empirical definition of a test's unidimen- 
sionality is a large first factor on the Item intercorrelation matrix 
(Porter, et al, 1978). A careful examination of achievement tests are 
especially crucial when siich tests are administered to low-achieving 
Title I students. > 

T^he general purposes of the present study are to investigate 
the factorial validity of the ebmprehenslve Teits of Basic Skills (CTBSj, 
Reading Comprehension Test^ Level 2, Form S, and to study the item 
difficulties in order to determine areas of strengths and weaknesses of 
sixth and seventh grade Title I students. Specifically, the focus of 
this study Is. on the following reseeirch questions involving Title I 
sixth and seventh grade students: 

1. Is the intercorrelation matrix which Is obtained from 
the. Items of the CTBS Readihq Comprehension Test worth 
factor analyzing? 

o 
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2. Does the GTBS Reading Comprehension Test have a strong 
factor structure? 

3. Is the GTBS Rielading Cbmprehehslon test unidlmehsional? 

4. Does one factor account for most of the ihtercorrelatlqns 
of the Items of the CTBS Reading Comprehension Test? 

5. What are the variance components of the CTBS Reading 
Cdmprehensidh Test? 

6.. What are the Item difficulties of the CTBS Reading 

Comprehension Test when administered to Title I students? 

7. Are there certain areas where Title I students appear 
tp.be especially weak when compared with CTBS norms? 

f^ethod . \ ^ ^ 

In a large southwestern school district, 581 ESEA Title i 

students (220 sixth grade students and 3(51 seventh grade students) were 

administered the CTBS, Reading Comprehension Test, Level 2, Form S, in 

the spring of 1979. All students were enrolled in a Title I program 

in which students were selected principally because of poor reading 

performance in st^^nines 1-3 on a norm-referenced -test. The ethnic 

background of participants is given in Table 1. 

Table 1. Ethnic Backgrounds of Participants 



Ethnlc-Background Percent , 

American Indian 6 
Black 16 
^' Hispanic 75 
Other 9 



idd 

The CTBS/S, Levels 1-4, is a battery of seven tests measuring 
three basic skills areas; Reading, Language and Mathematics (CTBS/S 
Test Coordinator's Handbook 1976). The skills areas were classified 
using Bloom's Taxonomy of Educa t i ona 1 Obj ec 1 1 ves . In the test 



development i efforts were made to reduce racial arid ethnic bias; The 
K-R 2B reliabilities at each grade level for vocabulary; cismprehensioh 
and total scores are almost all above .90 with standard erre^rs of 
measurement from .25 to 1.01 in grade equivalent units. Moreover, it ^ 
appeariBd that systematic procedures were followed in test development 
to ensure content validity. The CTBS/S Reading Comprehension Tests, 
Levels 1-4 are composed of 45 items and each item in Levels 1-4 contains 
a multiple choice Involving four alternatives. 

Before the statistical analysis, raw scores were converted 
to the CTBS/S Expanded Standard Score Scale. Calculations for the 
. present study Were performed with expanded standard s^cores unless 
otherwise noted. As the expanded standard scores form a normalized 
scale with assumed equal intervals, it was believed this metric was 
more appropriate for statistical analysis. This choice of a metric 
is in agreement with the technical advice to ESEA Title I evaluatOrs 
(Tallriadge and Wood 1976). When it was d^ired to convert an out-of- 
level test statistic up to the appropriate on-level statistic, the raw 
scores were converted to expanded standard scores and the desired 
statistic was computed with expanded standard scores. Then, the 
appropriate grade level table was referred to, and by means of the 
appropriate table, the expanded standard score was used to ascertain 
the grade level, raw score, percentile, stanine or grade equivalent. 
ResaTt& 

Participants in this study were selected for Title I because of 
poor reading ability. The guessing level is often high with Title I 
students. Moreover, research has shown that often random data can 
produce interpretable results when it is factor analyzed (Armstrong 

o 
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and Soelberg, 1968, Horn, 1967; Humphreys, Ilgehj HcGrath^ and Mdhtahelli, 
1969); Therefore, the first question to be answered in tifi?> study was 
whether the item iritereorrelation matrix was worth factor analyzing. 
Sorsuch (1976) presents Bartlett's (1950) test of significance for a 
correlation matrix: 

where n = number of individuals 

V " number of variables 
(R| = the determinant of the correlation 
matrix 

with df = V ^ 2 ^ 

-f 2 ^ 

The results for the sixth grade are x = 1^83, df = 990 and for the 
2 

seventh grade x = 2253, df = 990. These large chi -square values can 
be converted to a standard normal variable using this formula appearing 
in Edwards (1968, p. 67), and Glass and Stanley (1970, p. 520): 

> 

Z = -J 2 {x% 2(df) - 1 

^>'" 

. The chi -square values converted to standard normal 

variables are: for the sixth grade Z = 9.98 and for the seventh grade 
Z ^ 22.64. Since these two values were greater than Z = 1.96, a = .05,^ 
the two intercbrrelatibh matrices could legitimately be factored. 

To investigate the fa<^tor structure of the GTBS Reading Compr^- 
hehsibh Test when administered to Title I sixth and seventh grade 
students a principal components factor analysis was conducted followed 
by a Varlmax rotailbh. Principal components with eigenvalues less than 
1.00 were deleted from further study. Factor loadings greater than or 
equal to .30 wgre identified for further study. Since this was ah 
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expiofatdry factor analysis with more than twenty variables, the 
^ recotnmehdation of Nunnally (1978^ p. 418) was followed: . . it is 
strongly recommended that unitieS: be placed in the diagonal spaces and 
that component analysis be ahdertaken." Nunnally further recommends 
that the principal components analysis be followed by Varimax rotation 
(p. 418). The results of the components analysis are presented in 
Tables 2-5. . 



Table 2. Prlncijjal-eompbhehts Factor Analysis of the Correlation 
Ratrix of the Gbmprehensive Tests of Basic Skills^ 
Reading Gbinprehehsi bh Test Administered to ESEA Title I 
Sixth Grade Students* 



Factor 

. w 


Eigenvalue 


rerCen^ OT 

Variance 


Cumulative 
rercent ot 
Variance 


1 


4.42 


9.8 


9.8 


2 


2.08 


4.6 


14.4 


3 


1.85 


4.1 


18.5 


• 4 


1.67 


3.7 


22.3 


5 


1.59 


3.5 


25.8, 


6 6 


1.57 


3.5 


29.3 


7 


1.54 


3.4 


32.7 


8 


1.45 


3.2 


35.9 


9 


1.38 


3.1 


39.0 


10 


1.34 ^ 


3.0 


42.0 


11 


1.25 


" 2.8 


44.7 


12 


1.21 


2.7 


47.4 




1.18 


2.6 


50.1 


14 


1.14 


2.5 


52.6 




1.10 


2.4 


55.1 


16 


1.09 


2.4 


57.5 


17 


1.04 


2.3 


59.8 


18 


1.01 


2.2 


62.0 


19 


1 .01 


2.2 


64.3 



*N = 220 




Table 3. Factor Eeadings* ef the Principal Components Factor Analysis 
of the Gorrelatioh" Matrix of the Comprehensive Tests of 
Basic Skills, Reading Comprehension Test, Administered to ^ 
J ESEA Title I Sixth Grade Students (N = 220) 



\ Factor 

" " _._ 


Item 


Loading 


Factor 1 


7 


— - 
68** 




21 


46 


\ 


22 


43 




24 


38 


\ 


37 


-44 


\ 

___ — 
Factor 2 


2 


68 


5 


72 


Factor 3 


41 


80 




42 


4b 


- 

Factor 4 


12 


33 




19 


45 




21 


35 




23 


59 




26 


54 




28 


35 




29 


72 


Factor 5 


1 


64 






31 




12 


31 




16 


33 




36 


63 


Factor 6 


14 


57 




32 


76 


Factor 7 


18 


76 




24 


36 


Factor 8 


12 


-34 




17 ^ 


75 




30 


33 




38 


45 




42 


32 


Factor 9 


6 


49 




-8 


66 




11 


61 




16 


35 




45 


32 



Factor 



Item 



Loading 



Factor 10 


22 


-32 




40 : 


47 




43 


76 


Factor 1 1 


4 


7? 




10 


61 




28 


-36 


Factor 12 


27 


66 




3d 


42 






oo 


Factor 13 


9 


82 




In 








-4fi 


Factor 14 


35 


77 




36 


33 




45 


35 


Factciv 15 


15 


79 




39 


-37 




42 


30 


Factor 16 


. 3 


74 




13 


se 


Factor 17 


28 


48 




33 


80 


Factor 18 


19 


32 




39 


49 




44 


69 


Factor 19 


25 


P7 



*_ Only Factor loadings ^30 
** Decimal points are omitted ; 
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Table 4. Principal eompbhehts factor Analysis of the Gbrrelatlbn 
Hatrix of the Comprehensive Tests of Basic Skills^ 
Reading Cdinprehehsibn Test, Administered to ESEA Title I 
Seventh Grade Students* 



^ _ , Cumulative 

Percent of Percent of 



Factor 


Eigenvalue ^ 


, Variance 


Variance 


^ 1 ^' 




TOP 


1 9 Q - 


d 


O OK 




1 / .o 


o 

. o 


l./*t 


•3.7 


91 fi 
c 1 .0 


A 


1 AK 




9i Jl - 


p 


1 

1 • 


•J 1 


"97 d 

. CI 


6 


1 .34 


3.0 


30.9 ' 


7 


1.32 


^ 2.9 


• 33.8 


8 


1.26o' 


2.8 


36.6 


9 


1 .22 


2.7 


39.3 


lb 


1.20 


2.7 


42.0 


11 


1.17 


2.6 


44.6 


12 


1.13 


2.5 


,47.1 


13 


I.IG 


2.4 


49.6 


1# 


1.07 


2.4 


51 .9 


15 


1.04 


2.3 


^ 54.2 


16 


1.02 


2.3 


56.5 


17 


1-00 ■ ^ 


2.2 


58.7 


*N =361 
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Table 5. Factor Loadings* of Principal' Coffipdhents Factor Analysis 
of the Correlation Matrix<.(»f the Comprehensive Tests of 
Basic Skills, Reading Cjamprehensioh Test » ^Administered to 
ESEA Title I Severt^th Grade Students (N = 361) 



' Factor 



Item 



Loading' 



Factor 1 


3 ' 


32** 




5 . 


56 




6 


52 




7 


35 




8 


55 




11 


31 • 




14 / 


4d 




26 


63' 




21 


48 




22 


31 




26 


32 


Factor 2 


49 


73 


) 




41 


42 


71 


Factor 3 


- 
2 


-34- 




7 


32 




9 


54 




le 


68 


* 

Pa c tor 4 


38 


41 


3 




i 


18 


37 




3Z 


81 




45- 


39 


Factor G-" 


• 25 


47 




28 


- 55 




30 


64 




31 


35 




34 


55 




38 


^1 


Factor 6 


3 


-3D 




. 43 


68 




■ 44 


38 




45 


36 


Factor 7 


3 


37 




4 


32 




^ 6 


31 




13 


.67 . 




17 / 


51 


* Or\\s Factor Loadings 





Fact9r. 


Item 


Loading 


: Factor 8 


1 


: 81 




' 2 


315 


Factor 9 


19 


• 55 




27 


64 




39 

A 


46 


Factor 10 


7 ' : 


31 




33" 


71 




36 


51 


: Factor 11 


11 


40 




^ 18- 


33 




' ; 22 


£Q 




" 23 


73 


Factor '12 


25 


78 




26 


• 3D 




45 


41 


Factor 1 3 


14 


55 

<J<J 




15 


44 




19 


31 




29 




/ 


31 


57 


Factor 14 


_ 

7 


34 




9 


32 




35 


72 


Factor 15 


9 


33 




17 


32 




24 


68 


Factor 16 




77 




1? 


38 


Factor 17 


16 


47 




32 


79. 



Decimal points are omitted. 
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A strong factor structure was explained by Nunnally (1978, 

p. 418): 

By a strong structure Is meant one In which there are 
obvious groupings of the variables^ as evidenced In 
substantial cbrrelatidns among mOTbe^ of the groups 
and miich IbWer correlations between mernbers of 
different groups. In terms of factor analysis products, 
this means that each factor has some variables that ^ 
load mainly on it aldhe^ and there. are at least four 
tests with loadings above .5^0. Unless a factor is, at 
least that strong^ it would be best to ignore it. 

From an inspection of Table 3^ in which the factor loadings for 
the sixth grade students are displayed, only factors 1 and 4 approach 
the criterion for strong factor structure. In Table 5 in which the 
results for seventh grade students are presented, factor 1 clearly meets 
the criterion of strong factor structure and factors 5 and 13 approach 
the criterion. Thus, from the principal components analysis of the 
correlation matrices of the sixth and seventh grade Title I students, 
only factor 1 of the seventh grade students clearly attains the 
criterion for strong factor structure. 

Henrysson (1971, ppt 153-4) spoke of the use of factor analysis 

to determine the dimensionality of a test: A factor analysis 

is a check of the dimensionality of the^^t^^ If only 
one large general factor is found, except for some other 
very small loadings, the test is measuring only one main 
dimension. \ 

The principal components factor analysis extracted nineteen 
factors from the sixth grade intereorrelatioh matrix and seventeen 
factors friOT the sevehtfi grade tntercdrrelatibn matrix With the 
restriction that eigenvalues be greater than or equal to 1.00. These 

results suggest that the correlation matrix of the CTBS Reading 

i. • 



Compre hens Ion Test results can be partitioned Into many uncorrelated 

dimensions when administered to sixth and seventh grade Title I students. 

ft _ _ _ " _ 

With Title I students the CTBS Reading Comprehension Test did hot appear 

to be uhiaimehslohal i rather it appeared to be faetbrially complex. 

For construct validity* a test might be expected to have one 

factor account for most of the variance of the items. According to 

Ci-onbach (1971), 

For qpnstruct validity^ one would ordiiiarily desire 
sufficient coherence among items contributing to a score 
so that one factor would account for most of the Inter- 
correlation of items. But many constructs are exceptlbhs. 
If the construct Is conceived as a congldmerate^ one would 
expect the Items to fsll into clusters, (p. 470) 

Tests that by hypothesis are indicators of a certain 
construct are expected to show substantlalloadlngs 
on the same factors. When one of them loads on a 
second factor, this shows that the Indicator Is 
Impure, (p. 469) 

The first factor extracted fi^iM:he correlation matrix of the 

sixth grade students accounted for 9.8 percent of the total variance of 

the test. The remaining 54.5 percent of the variance of the test was 

distributed over the remainirigplS factors, the first factor extracted 

from the correlation matrix of thi s^hth grade students accounted for 

12.8 percent of the total test variance. The ronaining 48.9 percent of 

the variance was accounted for by the remaining sixteen factors. For 

both sixth and seventh grade students, the partitioning of variance among 

so many factors indicates a lack of coherence among items. Moreover, it 

appeared that one factor would not account for most of the intercorrelatlon 

of Items. 
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Kerlinger (1973, p. 478) presents the three variance cbmpohents 
of measurement theory: Total variance (V^) » a common factor variance (V^'^)^ 
specific variance (V^p) and error variance (V^). eommon factor variance 
is that vaY^iance that two or more measures share in common, and specific 
variance is that variance not shared by any other measure. The relation- 
ship between these components of total variance are specified in the 
following equation: 

t CO sp e 

this theory may be applied to the ^present study where the 
variances are proportions of the total variance. For the CTBS Reading 
Comprehension Test, the KR Formula 20 for sixth grade results was .75 
and for seventh grade was .BZ. The total common factor variance 
extracted from the sixth grade data was 64.3 percent and the^ seventh 
grade data was 58.7 percent. Since the reliability is equal to the 
common factor variance plus the specific variance, the basic equation 
partitioning the total variance into common factor variance, specific 
variance, and error variance can be written: 

S4jcth Grade Data 

100 % = 64.3% + 10.7% + 25% 

Seventh Grade Data 

100 % = 58.7% + 23.3% + 18% 

In the Comprehensive Tests of Beisic Skills, items are classified 
according to process (Blbbrri's Taxonomy) and content. Table 6 presents 
the item classification for the CTBS, Reading Comprehension Test, 

ERIC 



Level 2, Form S. Description of the process cateijorles are provided in 
Appendix A, and a desci^iption of the content is in Appendix B. 

the item difficulties (p-values) for the sixth and seventh 
grade were subtracted from the CTBS p-value norms, the averages of the 
differences were computed to get an overall estimate of the sixth and 
seventh grade differences with the norms. Within each of the seven 



process -content categories, the averages of the differences were computed. 
Tables 7 and 8 presents those differences. The higher the number, the 
greater the discrepancy with the national norms of that group of items. 



Table 6. Item Classification for Coffiprehinsive 
Tests of Basic Skills, Reading Compre- 
hension Test, Level 2, Form S 



Process-Content Category I tans 



6. 13, 
16, 21, 

22, 26, 
33, 34 

8. 20, 

23, 26, 

41, 45 

7* 9^ 14, 
15, 28 

2, 12, 

24, 38, 
44 

19, 3Q, 
32, 39 

U 10, 
11. 17, 
31, 37. 

42. 43 

3^ 4i 5. 
18, 25, 
27. 29. 
35, 40 
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1. Literal Recall 

2. Rewording 

3. Context Clues 

4. Main Idea 

5. Descriptive Words 

6. Conclusions 

7. Structure/Style 



Table 1. Item Dlffietilty Values of Students in Grades 6 and 7 Compared 
With CTBS National Norms 



r— - 


_ : 

Grade 


Grade 








Average of 


I tens 


6 


7 


Norms 


N-6 


N-7; 


Differences 


1 


' EE 

yP 


Ira 
04 


7C 
/O 


C 1 


1 c 


1** 4 


Z 


EE 

yy 


Oo 


fin 


CO 


1 9 


1^^ 


3 


oZ 


7c 


oi 
o 1 


Q 


1 7 


1^9 


4 




Af\ 

4U 


cc 
00 




9A 
£D 


9^ 7 


5 


EC 


/I 


od 


2D 


1 1 
1 1 


Ifi 7 

ID*/ 


6 


p3 


35 


07 

87 


OA 

Z4 


b 
2 


in b 


7 


33 


50 


77 

77 


bd 
39 


b7 
2/ 


bl c 
y 1 • 0 


8 




cc 
66 


82 


30 


1 c 
1 D 


CO • 0 


9 


30 


oo ri 

38 


' CO 

09 


bd 


b1 
3 1 


bi 

y*t • u 


IP 


33 


il o 

43 


D9 


OD 


9A 

CO 


9Q A 


1.1 


56 


C il 

64 


81 


OE ' 

2b 


1 7 


OA 

c\J • U. 


1? 


32 


il 3 

43 


43 


1 1 
1 1 




a 0 

4. t 


13 


56 


62 


77 


Ol 

2 1 


1 E 
1 D 


17 0 


14 


42 


57 


69 


o7 

27. 


1 O 

1 2 


17 7 


15 


40 


57 


7il 

/4 


34 


1 7 


00 4 
^y • 4 


1 c 
ID 


CO 

DO 


Do 


71 


1 0 


0 


- D • Q 


1 / 




En 


EO - 
DO 


1 0 


Q 
O 


10 7 


18 


oo 

Z3 


oc 
3o 


EE 
DO 


32 


1 D 


9^ Q 

CO • 7 


1 Q 


bu 


EC 
DO 


CO 

oy 




1 0 


1 R 

J y • O 


Zu 


C A 

50 


/o 


7Q 

/9 


23 


- y 


IR R 
J D* D 


91 
^ 1 


EO 

DO 




oo 
oo 


OC 
cO 


1 n 


1 c 7 


C.C. 




0? 


77 


y 1 


o 

o . 


IK 7 


^3 


EO 


RE 
DO 


74 




Q 

_ 3 


1 Q 

J y • 7 


9>i 
i4 


AH. 
40 


4/ 


00 


IQ 

17 


lO 


Iff d 

LP • H 




^9 


57 


Oc 


90 


9C\ 


9A 9 




04 


oO 


oD 


OO 
22 


c 

-0 


19 1 




OE 

25 


33 


56 


3 1 


OO ' 

23 


OK n 

20 .U 


z8 


CA 

59 


CA 

69 


70 

/2 


1 o 
13 


3 


C fi 
_D«D 


Z9 


Zo 


MO 

42 


CO 

62 


oc 
3d 


OD 


OC 1 
20 • 1 


- 30 


E5 

53 


04 


7E 

Id 


OO 

22 


11. 


1 C 0 
10 • 2 


31 


33 


3/ 


HA 

64 


Ol 

31 


07 

2/ 


OQ C 
29. 0 


32 


14 


1 n 

19 


35 


oc 

Z5 


on 
20 


ol o 
Z1.9 


33 


33 


41 


ca 

64 


31 


03 

23 


OC n 

Zo.u 


34 


~M O 

42 


n o 

48 


cc 

65. 


OO 

Z3 


17 


TO 3 

19.3 


35 


30 


46. 


61 


31 


14 


20.4 


36 


27 


36 


59 


32 


23 


26.4 


37 


19 


22 


32 


13 


10 


11.1 


38 


26 


39 


66 


40 


27 


31.9 


39 


21 


34 


58 


37 


24 


28.9 


40 


30 


41 


51 


21 


10 


14.2 


41 


35 


42 


65 


30 


23 


25.7 


42 


35 


43 


56 


21 


13 


i6;o 


43 


29 


35 


54 


25 


19 


21.3 


44 


26 


37 


58 


32 


21 


25.2 


45 


22 


24 


48 


26 


24 


24.8 



Item difficulties (p-values) have been cdhverted to percentages. 



Table 8. The Average of the Mean Differences Computed 
Within Each GTBS Content Category 



Cbriteht Category 


Mean Differences 


1 . Literal Recall 


• 15 ^ 


2. Rewording 


•22 


3. Context Clues 


.23 


4. Rain Idea 


.19 


5. Descriptive Words 


.20- 


6. Conclusions 


•19 


7. Structure/Style 


.21 



In Table 7, the average of the p-values ^ifferenets are computed 
to identify categories which are easier or more difficult for Title I 
sixth and seventh grade students. This data is presented as a bar graph 
in Figure 1. The Literal Recall Category was the (easiest when compared 
with national norms. The average p-value difference was .15. this 
category is jfehe recall of facts and details explicitly stated . in a 

0 

passage. 
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Figure 1. Average mean differences of combined sixth and seventh t 
grade students. 
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The categories of moderate difficulty were Main Idea (.l9r> . 
beiscriptlve Wordis (.20) and tdncluslons (.19). Main Idea refers to 
such skills as identifying the topic of a passage, restating or summarizing 
the central thought of a passage and Identifying the authors purpose. 
Descriptive Words include defining the physical attributes' of an objective 
or person presented in a passage and analyze emotions experienced In the 
passage. Conclusions Include skills such as drawing c«:^nclus1on^ based " 

oh facts stated or implied in the passage and perceived cause and effect 

.1 

relationships. 

The difficult categories when coffipared to national norms were 
Rewording (.22), Context Clues (.23) and Structure/Style (.21). Rewording 
is to give facts and details in a passage and to answer questions by 
choosing paraphrased or reWQrded material. Context Clues refers to 
detennihing word meaning from the context in which a word is used. 

C i 

structure/style 'includes the identification of the writers use of words, 
figurative language and the interpretation of symbols. Moreoyer, this 
category refers to understanding structured devices that help convey 
" meaning such as punctuation, function words or tense. 

the CTBS content-categories were not ascending in the average 
-difficulties within categories. RatheV one category, titeral Recall 
was noticeably easier for combined sixth and seventh grade Title I 
students. The other six categories were classified as moderately difficult 
to difficult for Title I students. ^ 



. A z-statistic wa is ^computed to test the sighificafice of the 
difference between two Independent pfoportiiDhs (Glass and Stanley^ 1970.^ 
pp. 324-326). This statistical test was u|dertaken to identify those 
p-values that were^sighificantly different between the sixth and seventh 
grade students. The results are presented in Table 9. 



Table 9. edmparisoh of Item Di'ffi'cu1ty''Va1ues of the Sixth and Seventh 
Grade Title I Students 
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Grade 


Grade 




Items 


6 


'7 


z-value 



1 

2 
3 
4 
5 
6 
7 
8 
9 

Id 

11 
12 
13 
14 
15 
16 
17 
18 
19 
29 
21 
22 

23 

24 

25 

26 

27 

28 

29 

3D 

31 

32 

33 

34 

3B 

36 

37 

38 

39 

4d 
41 

42 ; 
43 
44 
45 



* p < 
** p< 



05 
01 



.55 
.55 
.62 
.46 
.56 
.63 
.38 
.46 
.30 
.33 
-.56 
.32 
.56 
.42 
.40 
.58 
;43 
.23 
;5d 
;5d 
.58 
.46 
.52 
.46 
;29 
.64 
;25 
.59 
;26 
.53 
.33 
.14 
.33 
.42 
.30 
.27 
.19 
.26 
.21 
.30 
.35 
.35 
.29 
.26 
.22 



.64 
. ;68 
;72 
iiB 
.71 
;85 
;5>3 
i66 
i38 
;43 
;64 
.43 
.52 
.57 
.52 
.68 
.50 
.36 
.56 
.70 
.73 
.69 
.65 
.47 
.27 
.80 
.33 
.69 
.42 
.64 
.37 
.19 
.41 
.48 
.46 
.36 
.22 " 
.39 
.34 
.41 
.42 
.43 
.35 
.37 
.24 



Si 



-2.09* 

-3.06** 

-2.^** 

1.38 
-3.58** 
-5.94** 
-2.73** 
-4.61** 
-1 
-2. 
-1 
-2. 



-3. 
-2. 
-1 
-3. 



90* 
.32* 
.86* 
,56** 
-1.39 
-3.41** 
.85** 
37** 
.59 
,18** 
-1.37 
-4.70** 
-3.64** 
-5.34** 
-3.02** 

- .23 
.51 

-4.15** 

-1.98* 

-2.39** 

-3.77** 

-2.55** 

- .95 
-1 .50 
-1.87* 
-1.37 
-3.70** 
-2.17* 

- .84 
-3.11** 
-3.24** 
-2.58** 
-1.62 
-1.85* 
-1.45 
-2:65*5* 

- .54 
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Using a one-tailed test df significance twenty-foar Items 
differed significantly between sixth and seventh grade students 
(p < .01 )» eight items difference significantly between sixth arid 
seventh grade itaderits (p < .05) and thirteeri items did hot differ 
sigriificaritly. Items that were different between sixth and seventh 
grade Title I studerits 3jd not, fall into easily interpretable categories. 
Bispussiori 

Using Bartlett's test. It was detehnihed that the Title I sixth 
and seventh grade item iritercorrelation matrices were worth factor 
analyzing. This was a necessary preliminary to this factor analytic 
investigation of test rfestilts of low-aqhleving students. Clearly the 
GTBS Reading ebmprehehsioh Test did hot have a strong factor structure 
wheh administered to Title I sttidehts. Rather than being unldimensional » 
the test results appeared to be multidimensional, the test results 
were factbrially eomplex. One factor did not accouht for most of the 
ihtercbrrelatiohs among the Items. Large specific and error components 
were found In the test results, this study presents evidence that when 
the CTBS Reading Comprehension test Is administered to low-achieving 

Title I students, the results are not measuring a single, unidimensional 

\ . 

construct. 

_ ^ _"_ \_ _j_ _ 

The error variance Is considerable in these test results. Since 

principal components analysis totalllj^ partitibris the test variance, riot 

just the commori factor variarice, w1th\^the oril^ restrictiori that eigerivalues 

be greater than or equal to 1.00, the error arid specific variarice wis 

probably intermixed with factors and tl^(is accounted for the large number of 

factors . 

, ■\ 

\ o6 



This study suggests that the results from testing Title I 
studerits with standardized tests should be interpreted cautiously, 
the tptaV score for each student is far from representing any one 

construct and the error variance in the testing is considerable. One 

, - - - _ - j - - - - ________ ^ __ 

method suggested for reducing error variance is out-of-level testing. 

It should be pointed out that the seventh grade students were tested 

out-of-level, that is, brie level below that reconmended by the publisher, 

and the sixth grade studerits were tested bri level. The factor analytic 

results were not noticeably different for these two groups. One might 

consider replicatirig this method with more extreme groups. 

In summary, two methods of scrutinizing test results is with 

factor arialysis and item analysis. Researchers should not rely simply 

oh interpretatibris of total scores, means, and mean differences. In 

order to more fully understand the mechanisms of the test, one should 

explore the test results in a variety of ways. Two of those ways are 

factor analysis and 1t^ analysis. 
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. APPENDIX A 



Process Categories Adapted from Bloom's Taxonomy Used for Glassifyihg 
Items in the Comprehensive Tests of Basic Skills, Reading Comprehensiori 
Test ' • \ 



Recognize, or recall and apply^ facts ^ theories, laws, concepts, 
structures, methods, and formulas previously learned. 

Translation 

Transform concepts, terms, and symhols into equivalent forms, as 1h; 
interpreting graphically presented data. 

Interpret tj on^ 

Understand facts and concepts In, and make inferences from ^ written 
material. Associate data and ideas and comprehend relationships 
stated or implied in material Summarize and restate significant 
themes or concepts. 

AppHcatibh 

Apply prior knowledge of facts or technlgues to answer questions and 
solve prbbl ens. Utilize previously learned research skills and 
techniques to determine function or meaning of new material. 

Analysis . 

Apply formal logic to the breakdown of material into organizational 
patterns or principal components; the determination of validity, bias, 
toriei the hierarchical arrangement of facts; the cognizance of total 
problems. 
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APPENDIX B 



Bescription of Content Categories for the Comprehensive Tests of Basle 
Skills, Reading Comprehension Test 

l-iteral RecaTl 

Recall facts and details explicitly stated in a passage. The student 
is required to answer "who," "what," "where," and "when" questions. 



Rewordijig 



_ Given facts and "details 'in a passage, answer questions by ehoosind 
/ ... paraphrased or- reworded*material . 

Context Cities 

Determine word meaning from the context in which a word is used. 
Main idea 

_. __ __ _ - \ _ . _ _ . i 

Identify the topic of a passage, restate or summarize the central 
thought In a passage r Identify- the author^s purpose In writing the 
passa.ge,vder1ve a'Tessbh^^^o^ from the passage, and seledt the 

best t1;tro>for a passage. \ 

Descriptive Words 

Define the physical attributes of an object or person presented In the 
passage, analyze.,.efflotfohs experienced in the passage, or decide what 
quality best characterizes cj, person. 



Conclusions 

Draw conclusions based on facts stated or implied in the passage or 
perceive cause-and-effect.relatiohships between events or ideas. 

St r u cture/S tyle 

Identify the writer's use of words, including figurative language^ to 
evoke a feeling or create art image, interpret symbols* and understand 
the writer's point of view and method of conveying meaning, such as 
punctuation, function words, tense. 
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